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1.  Introduction 

The  purpose  of  this  paper  is  to  sketch  an  application  of  some  ideas  in  the 
mathematical  theory  of  control  processes  to  biological  phenomena  such  as  in¬ 
stinct,  learning,  curiosity,  adaptation,  and,  finally,  consciousness.  We  shall  em¬ 
ploy  the  language  and  methodology  of  the  theory  of  dynamic  programming. 
Detailed  accounts  of  the  mathematical  ideas  will  be  found  in  [1],  [2],  [3].  For  a 
quite  different  approach  to  consciousness,  see  [6]. 

2.  Control  processes 

We  begin  with  the  idea  of  a  process.  Consider  a  system  described  by  a  point  p 
in  a  space  S.  Let  T(p)  be  a  transformation  with  the  property  that  px  =  T{p) 
belongs  to  S  whenever  p  is  in  S.  We  call  the  pair  [p,  T(p)]  a  process.  More 
precisely,  this  is  a  particular  description  of  a  process. 

When  the  transformation  is  repeated,  yielding  a  sequence  of  states,  px,  P2,  •  •  •  , 
where  px  =  T(p),  p2  =  T(px),  •  •  •  ,  we  call  it  a  multistage  process.  This  is  an 
abstract  version  of  a  dynamic  process. 

Assume  next  that  T(p)  is  replaced  by  a  transformation  of  the  form  T(p,  q) 
having  the  property  that  for  any  p  in  S  and  any  q  in  a  decision  space  D,  the 
point  pi  =  T(p,  q)  is  in  S.  A  choice  of  vectors  (decisions)  qx,  q2,  •  •  •  ,  then  yields 
a  sequence  of  states  px  =  T(p,  qx),  p2  =  T(px,  q2),  •••  .  We  call  this  a  multistage 
decision  process.  It  is  also  an  abstract  version  of  a  control  process.  From  the 
mathematical  point  of  view,  control  and  decision  processes  are  equivalent;  see 
[2],  [3],  . 

A  determination  of  the  <?»  may  be  effected  by  maximizing  a  criterion  function 
which  depends  on  the  history  of  the  process  K  =  K{p,  px,  •  •  •  ’,qx,qi,  •  •  •)• 
In  many  important  cases  this  has  a  separable  structure  K  =  kx(p,  qx)  + 
k2(px,  q2)  +  •  •  •  ,  in  other  words  an  accumulation  of  single  stage  effects.  A 
criterion  function  is  a  measure  of  the  effectiveness  of  a  control  process. 

The  maximizing  qi  will  be  functions  of  the  states  px,  px,  •  •  •  .  In  the  most 
important  cases  the  which  maximizes  depends  only  upon  the  present  and  past 
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history  of  the  process  gt-  =  g,(p,  ph  •  •  •  ,  p,_i),  and  frequently  only  upon  the 
current  state,  g,-  =  g,(p;_i). 

Let  us  call  any  function  of  this  type  a  policy,  reserving  the  term  optimal 
policy  for  a  policy  which  maximizes  K.  The  study  of  control  and  decision  proc¬ 
esses  may  then  be  considered  to  be  the  study  and  effective  determination  of 
optimal  policies.  Unfortunately,  in  many  of  the  control  processes  of  greatest 
significance  either  no  criterion  function  exists  or  there  are  too  many  of  them. 
(The  criterion  function  is  a  measure  of  the  sequence  of  decisions.)  This  makes  the 
application  of  mathematical  theories  such  as  the  calculus  of  variations  and 
dynamic  programming  quite  difficult;  see,  however,  [5]. 

Nevertheless,  the  concept  of  policy  remains  meaningful.  Furthermore,  the 
powerful  and  flexible  theory  of  simulation  can  be  used  to  study  the  complex 
processes  associated  with  animate  and  human  systems;  see  [3]. 


3.  Instinct 

Let  us  now  identify  instinct  as  a  policy,  a  policy  which  controls  the  behavior 
of  an  organism  in  a  particular  situation.  We  consider  instinctive  behavior  a 
precise  automatic  response  to  a  signal  or  stimulus  acting  on  an  organism.  The 
evolutionary  value  of  instinct  is  clear  since  in  critical  circumstances  there  may 
not  be  time  for  conscious  behavior.  The  response  must  be  preprogrammed  to 
ensure  survival. 

The  point  we  wish  to  emphasize,  however,  is  that  instinct  is  not  solely  a  low 
level  intellectual  activity.  What  seems  to  be  the  case  is  that  there  are  levels  of 
instinctive  behavior,  intermingled  with  conscious  behavior.  We  shall  return  to 
this  point  below. 


4.  Levels  of  control  processes 

When  the  original  system  does  not  cope  in  a  satisfactory  style,  survival  of  the 
species  depends  upon  the  development  over  time  of  control  mechanisms  to  im¬ 
prove  the  performance.  At  the  lowest  level  these  are  feedback  control  devices. 
These  control  systems,  however,  themselves  require  supervision  and  modifica¬ 
tion.  Gradually,  then,  we  see  the  development  of  control  systems  for  control 
systems,  and  so  on.  The  operation  of  any  complex  system  requires  a  hierarchy 
of  control  systems. 

We  can  identify  this  hierarchy  of  control  systems  with  levels  of  consciousness. 
Indeed,  if  we  wish  to  discuss  consciousness  in  a  meaningful  fashion,  we  must 
utilize  the  concept  of  levels  of  consciousness.  Similarly,  to  study  thinking,  we 
must  consider  levels  and  kinds  of  thinking.  A  confusion  of  levels  leads  to  para¬ 
doxes  and  other  difficulties;  see  [4],  [7]. 
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5.  Instructions 

Let  us  pursue  this  subject  of  hierarchies  in  another  direction.  Suppose  that 
an  organism  contains  certain  types  of  mechanisms  for  conveying  instructions. 
Some  of  these  instructions  may  well  be  instructions  for  issuing  instructions,  and 
so  on,  again  a  hierarchal  concept. 

An  error  in  one  kind  of  instruction  is  thus  seen  to  be  far  more  critical  than  an 
error  in  another  type  of  instruction.  An  error  in  the  instructions  for  issuing  in¬ 
structions  will  lead  to  an  infinitely  greater  number  of  errors  than  an  inaccurate 
instruction  which  leads  to  the  development  of  one  faulty  organ,  for  example. 
We  see  then  that  numerical  probabilities  of  mutation  are  not  inherently  mean¬ 
ingful.  We  must  take  the  structure  of  the  organism  into  account  and  examine  the 
responsiveness  of  the  structure  to  a  change  in  one  component.  These  are  ques¬ 
tions  of  mathematical  stability  and  sensitivity. 

6.  Uncertainty 

So  far  we  have  considered  only  deterministic  processes,  processes  where  cause 
and  effect  is  assumed  to  hold.  Let  us  now  consider  processes  where  uncertainty 
plays  a  major  role. 

One  way  to  construct  mathematical  models  of  uncertainty  is  to  introduce 
random  variables.  This  leads  to  the  concept  of  a  stochastic  transformation 
T(p,  q,  r).  The  point  px  =  T(p,  q,  r )  belongs  to  S  whenever  p  is  in  S,  q  is  in  D 
and  r  is  a  random  variable  in  R;  see  [2]. 

A  choice  of  a  sequence  qX)  q2,  •  •  •  ,  and  a  selection  of  random  variables 
ri,  r2,  •  •  •  ,  leads  as  before  to  a  sequence  of  states  pi,  p2,  •  •  •  .  An  optimal  policy 
may  be  determined  in  a  process  of  this  nature  by  maximizing  the  expected 
value  of  a  criterion  function. 

This,  however,  takes  care  of  only  first  level  processes.  There  are  far  more 
complex  levels  of  uncertainty.  In  the  foregoing  we  assumed  that  the  probability 
distributions  for  the  random  variables  were  known.  Frequently,  this  is  not  the 
case.  Instead,  we  may  possess  some  initial  clues  as  to  the  nature  of  the  uncer¬ 
tainty  and  then  proceed  to  discover  more  of  the  structure  of  the  process  using 
observation  of  the  events  that  occur.  We  call  this  an  adaptive  control  process, 
and  equate  this  operation  with  learning.  One  instinct  associated  with  learning 
is  curiosity.  There  are,  however,  levels  of  learning.  We  learn,  we  learn  how  to 
learn,  and  so  on;  see  [5]. 

7.  Higher  level  instinct 

A  mathematical  technique  for  studying  adaptive  control  processes  is  the  the¬ 
ory  of  dynamic  programming.  In  animals,  however,  it  seems  that  adaptive 
control  involves  higher  level  instincts.  If  so,  this  plausibly  explains  why  it  is  so 
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difficult  to  carry  out  mathematical  studies  of  pattern  recognition,  language 
translation,  and  human  communication.  If  these  are  instinctive,  they  are  not 
based  upon  the  mathematical  principles  of  the  last  5000  years,  but  instead  upon 
techniques  developed  by  evolutionary  selection  over  hundreds  of  millions  of 
years. 

In  other  words,  we  possess  in  our  brain  very  complex,  genetically  determined, 
internal  mechanisms  specifically  designed  for  particular  tasks.  Existing  mathe¬ 
matical  methods  and  computers  cannot  yet  compete  with  these  consequences 
of  selective  breeding. 


8.  Theory  of  types 

It  is  apparent  that  what  has  proceeded  has  been  considerably  influenced  by 
the  Russell  theory  of  types  and  the  Liouville-Ritt  theory  of  elementary  func¬ 
tions.  See  the  expository  discussion  in  [7],  as  well  as  [4]  for  an  analysis  of  one 
type  of  humor  in  terms  of  paradoxes. 
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